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Violent extremist groups are currently making intensive use of Internet fora for recruitment to terrorism. 
These fora are under constant scrutiny by security agencies, private vigilante groups, and hackers, who 
sometimes shut them down with cybernetic attacks. However, there is a lack of experimental and formal 
understanding of the recruitment dynamics of online extremist fora and the effect of strategies to control 
them. Here, we utilize data on ten extremist fora that we collected for four years to develop a data-driven 
mathematical model that is the first attempt to measure whether (and how) these external attacks induce 
extremist fora to self-regulate. The results suggest that an increase in the number of groups targeted for 
attack causes an exponential increase in the cost of enforcement and an exponential decrease in its 
effectiveness. Thus, a policy to occasionally attack large groups can be very efficient for limiting violent 
output from these fora. 

Emerging technologies that permit data collection at unprecedented scale can reveal patterns of individual 
and group behavior 1 5 and they have propelled the development of data-driven mathematical models of 
human conflict. These include models of insurgent and terrorist activity 610 , global trends in violence across 
human history 1113 , cultural and ethnic confrontation 1417 , and the structure and dynamics of criminal organiza- 
tions 18 " 20 . 

Here, we turn our attention to violent extremist groups that make intensive use of Internet fora for recruitment, 
illegal financing, internal communication and, in fully radicalized individuals, operative support of real world 
violent activity 2124 . For the past decade, extremists have been using online networks to socialize their members 
and to reinforce aspects of their radical culture 25,26 . These networks diminish the sense of isolation for members of 
extremist groups and give them access to a global community that enjoys the legal and religious approval of the 
leaders of their movements 27 . They also facilitate the acquisition of articles, books, and audiovisual material with 
radical content that are created in clandestine or underground circles. Without access to these networks, obtain- 
ing material like these would be much more difficult 28 . 

Online fora are particularly important for extremist groups, as they play a key role in the terrorist recruitment 
process. These platforms have not only become a common medium for extremist organizations to spread their 
propaganda and obtain funding, but they have also transformed into the primary arena where the process of 
violent radicalization takes place 2 '. Extremist fora allow users to reinforce their identities as militants through 
virtual activism and help to create a hierarchy within the group that can be easily monitored by the group leaders. 
In fact, it is possible to detect specific participation rules in these fora that allow users to improve their reputation 
within the group and their status within the community 30 . Those users who achieve the highest status may then be 
recruited to engage in real world operations 2124 . 

Although it is well documented that recruitment into online fora can lead to recruitment into terrorist 
organizations, it is important to emphasize that these processes are not the same. We are not claiming that online 
fora are a proxy for real world terrorism. Rather, we are claiming that they are an important entry point for some 
people who later engage in real world acts of violent extremism, and it is therefore important to understand how 
these fora operate and how we might act to reduce their effectiveness. 

Several thousand websites support and promote extremist discourse, but not all have the same impact. A very 
small number of sites exert a very large amount of influence because they are the only ones to directly receive 
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Table 1 
Forum 


List of online fora analyzed. Forum names and URLs 
Starting recording date 


are available from the authors on request 

Last recording date Total 


number of days recorded 


rorum 1 


uo/ U/ / zUUo 


n~7 /m /9n l l 

U/ /Uj/iU 1 1 




Forum 2 


U 1 / U 1 / zuuo 


H9 /9^ /9fl 1 1 
UZ /Zj/ZU 1 1 


7 


Forum 3 


f)9 /9T /9f)l O 


OA /9T /9fl 1 1 
UO/ ZJ/ ZU 1 1 


AA 


Forum 4 


08/02/2008 


07/03/201 1 


46 


Forum 5 


02/25/201 1 


07/03/201 1 


39 


Forum 6 


02/25/201 1 


07/03/201 1 


39 


Forum 7 


01/17/2009 


03/23/201 1 


21 


Forum 8 


10/29/2008 


07/03/201 1 


49 


Forum 9 


02/05/201 1 


07/03/201 1 


1 1 



material prepared by extremist groups 31 36 . On these fora one can 
find ideological diatribes, interviews, and audio and video recordings 
of recent acts of violence. The other sites simply repeat and amplify 
the content disseminated by the most prominent extremist fora. 

Extremist Internet fora operate in a hostile environment as their 
growing importance has caused them to come under scrutiny in 
recent years. Indeed, the conventional wisdom is that the longevity 
of an online community is inversely proportional to its importance in 
the radical universe. Organizations dedicated to monitoring extrem- 
ist groups on the web note that 80% of the top websites to emerge 
between 2002 and 2004 have disappeared because they have been 
unable to withstand continuous harassment by security agencies, 
private groups, and hackers 37,38 . These attacks are designed to infilt- 
rate and boycott extremist fora 39 and they have also succeeded in 
provoking mistrust and mutual accusations between rival groups 21 . 
As a result, administrators are suspicious of new users who are too 
active or who express a clear interest in contacting forum members in 
person. Some platforms have implemented stricter self- protection 
measures such as making their entire content available only to regis- 
tered users, closing admission to new members, or accepting only 
new users who have been endorsed by a current one. 

Actors seeking to disrupt recruitment to extremist fora face several 
technical questions 40 43 . Which fora should be targeted for an attack? 
What strategy should be used? Given that these attacks require costly 
human resources to be implemented, it is important to assess the 
impact of various strategies, and to do this we need data and a model 
that will help us to better understand how fora react when they are 
under stress. Elucidating these questions will shed light into an 
important debate: Should terrorist groups be allowed to use the 
Internet, or should they be banned from it 44 ? 

In order to analyze the potential impact of cybernetic attacks, we 
recorded the number of members registered from 10 different fora, as 
reported automatically by the forum platforms (see Table 1, Table 2, 
and Figure 1 for the list of forums analyzed and monitoring statist- 
ics). Recordings were performed at 12 AM GMT. The data collection 
started 1/1/2008 for the Forum 2, which is the earliest forum that we 
were able to detect, and spanned the next 4 years for the rest of the 



Table 2 Monitoring statistics for the fora analyzed. The statistics 
are calculated for the full observation period and all fora, with the 
aim to give an intuition on the large variability in our ability to 
collect data across different fora 

Number of Fora 9 


Starting recording date 


1/1/2008 


Final recording date 


7/3/201 1 


Total number of days recorded 


296 


Min value 


13 


Max value 


14,135 


Mean value 


5778.1 


Median 


4912 


Standard deviation 


4228.1 



fora. The number of measurements per forum ranges from 7 to 49, 
and the sampling rate was constrained by the availability and access- 
ibility of each forum. In Table 2 we provide the details of the data 
collected for each forum and overall statistics showing, among other 
things, the skewness of the distribution (notice that the mean number 
of members is significantly larger than the median). Forum 10 is not 
shown here, as it was discarded from our analysis for reasons dis- 
cussed below. 

The reported information was publicly accessible without login or 
participation required. No further checks on the validity of these 
reported figures were performed, so we cannot rule out systematic 
biases introduced by the forum administrators (e.g. multiple regis- 
trations by the same users, counting of inactive users, etc). During the 
observation period we did not detect any deliberate efforts to stra- 
tegically misrepresent the data. However, given the possibly quite 
large incentives for misrepresentation, further work on the assess- 
ment of data veracity is certainly needed. We also cannot rule out 
problems arising from the left-censoring of our data, as we lack 
information about activity prior to the detection of a forum. 

Although the observation period for the 10 fora spans 4 years, we 
were unable to record information on a daily basis, as we would find 
the fora inaccessible on many occasions - mostly de-activated with 
no website connectivity. The time-span for which these fora would be 
de-activated varied from days to months. Even though we may sus- 
pect that the fora under study were shut down as a result of an attack, 
we do not have direct evidence of this attack, and cannot establish a 
causal link. 

In selecting the sites for analysis, we focused attention on fora that 
constituted the "inner circle" or core of the violent extremist pres- 
ence on Internet. These fora 1) served as primary sources of dissem- 
ination of propaganda produced directly by extremist organizations, 
2) had users that were implicated in real-world acts of terror, and 3) 
had the largest following and influence in the extremist cyber- 
community 31 36 . In addition, we only included a particular forum if 
we detected at least one instance where sharing of text or multimedia 
material prepared by extremists took place during the observation 
period. These fora also contained normal discussion and sharing of 
non-extremist information among members. 

We excluded from our analysis "secondary" fora that hosted rad- 
ical content copied from other sites because they typically attract very 
few users. This also lead us to discard one of the ten fora under study, 
run from Spain prior to August 2011. During its three-month exist- 
ence, it only managed to attract 13 registered users despite its more 
than 7, 800 discussion topics and 10, 000 posts from other sites, 
which were uploaded daily by the site administrator 45 . 

In Figure 1 we show the rate of growth for each forum, and given 
the different initial start dates we also show these curves fitted to 
reduce the Euclidean distance to Forum 8, which had the most 
observations. These data suggest that fora exhibit a first phase of 
exponential growth followed by a saturation phase in which they 
continue to grow exponentially but at a much slower rate. This con- 
trasts with the constant exponential growth (with probably a very 
large cutoff established by the size of the susceptible population) one 
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Figure 1 | (TOP) Number of members participating in the 9 extremist 
fora under study. Note that initially the fora grow exponentially and then 
saturate growing at a exponential yet much slower rate. (BOTTOM) Fora 
time shifted to match the early stages of the development of every fora and 
gain visual intuition on their dynamics. In particular, we aligned the early 
stages of the all fora with Forum 8, with the exception of Forum 5, which 
was aligned with Forum 9 due to a similar volume of members. 



might expect if members were recruited and allowed to join without 
restrictions (as in the classic susceptible-infected epidemiological 
model 46,47 ). Case studies suggest that extremist fora control growth 
by limiting the number of participants 48 . 

In what follows, we describe a dynamic characterization for the 
evolution of one forum, and then the population dynamics of 
multiple interacting fora. Our goal is to provide a plausible explana- 
tion for the empirical observations and to understand the impact 
of persistent disruptive attacks on the evolution of these online 
communities. 

Results 

We start by denoting N as the total population that is susceptible to 
becoming a forum member and let 7t 10 be the rate of becoming a 
forum member. Within the forum itself there are several levels of 
status. We are interested in the number of users at each level, so let 
Xj = Xj(t) be the number of active members at level i. We assume that 
there is a process for advancement that allows users to go from one 
level to another. The rate of transition between levels is given by the 
transition probability matrix Tty which denotes the probability per 
unit of time of moving from level j to level i. 

For simplicity, we assume the transition probability matrix can 
only move upwards, hence 7Cy = 0 for j S It may be the case that 



some users exit the process at different points through withdrawal 
(i.e. de-registration), inertia, lack of Internet access, and so on. 
However, in our model, the probability of exit is absorbed into the 
transition probabilities from one level to the next one, where a prob- 
ability of disengagement would yield a smaller probability of trans- 
ition upwards. We also assume that the number of law enforcement 
and intelligence agents inhabiting these fora are much smaller than 
the number of genuine members, as otherwise it would raise aware- 
ness of the forum administrators and compromise their infiltration 
operation. 

Suppose that there are L levels within the forum. At the highest 
level an individual may become fully recruited to engage in real world 
operations (O) or to become a recruiter (R). We denote O and R the 
output of the model and recruiters, respectively, produced by the 
forum. Recruitment is the bottleneck of the radicalization process, 
as the number of recruiters determine the change rate of X, R, and O. 
This is implicit in the evolution of X L , as in this equation, the mem- 
bers leaving the forum at level L depend on R, not on X L . 

The dynamics of the populations within a particular forum can be 
written as 

dXi 

■ =Ki fi N—K 2t iX l , 



dt 

dXj 
~dt 
dX L 
dt 
dR 
~dt 
dO 
~dt 



iXl-i ~Po,rR ~Pr,rR, 

=Pr,rR, 
=Po,rR, 



(1) 



where 2 < f < L - 1, andp R R , and p 0>R denote the transition rates to _R 
and O, respectively. In these equations we assume that the number of 
recruiters is smaller than the number of members because there is a 
bottleneck in the conversion from being a forum member to being a 
recruiter (R) or engaging in real life terrorism (O). We also assume 
that the total population is much larger than the number of users in 
the forum N » X. 

The model itself is a variation of a mass-action compartment 
model familiar in mathematical epidemiology 46,47 . Conceptual sim- 
plicity drives the choice of this model, though we must stress the 
caveat that changes in psychological state or activity may not neces- 
sarily be similar to changes in disease status. 

Full observation of all the states L, O, -R presents ethical, experi- 
mental 49,50 , and legal complications 51 , as it requires interaction with 
the Forum participants for long periods of time. Thus, we present a 
simplified version of the model where we merge all the levels of the 
fora into one: X = Xi + X 2 + . . . + X L . Thus, we can rewrite the 
previous equations as 

dX 

= n U oN-(p RM +p 0 ,R)R, 



dt 
dR 
dt 
dO 
~dt 



=Pr,rR, 
=Po,rR, 



(2) 



The initial conditions are X(0) = 0, R(0) = Rq > 1, O(0) = 0. The 
main output of this model is dOldt = po, R R and note that we are 
assuming that there is a fixed number of susceptible individuals, N, 
who may become members of the forum. We also need to take into 
account the fact that the success of a forum may depend on its 
popularity (for example, larger fora may be able to attract more 
new members). Thus, we make the assumption that n 10 has an 



SCIENTIFIC REPORTS | 3 : 1544 | DOI: 10.1038/srep01544 



3 



explicit dependence on the forum size as follows, 7i 10 (X) = p N (X)X, 
with p N (X) > 0 for X > 0. 

Next, we address the possibility that as fora become more popular, 
they are more likely to draw the attention of anti-extremist author- 
ities and other agencies, and therefore may be attacked and shut 
down. We can model this by assuming that, when forum population 
(X) reaches a threshold 0, the forum is attacked and disabled, causing 
all its variables (X, R, O) to go to 0. 

Forum administrators may try to balance two forces. On the one 
hand, they are aware that larger fora are more likely to be attacked, so 
they may try to control the number of users on the forum. On the 
other, they may want to maximize membership, making X as large as 
possible (i.e., just below the threshold 0). 

For simplicity, suppose the transition rates to become a recruiter 
or terrorist are constant such that pnn=p° R >0 and po,u=pQ>0. 
Assume X* is the target value. Let X(r*) = X*, where X* is the target 
control condition for the number of members of the forum. The 
stationarity condition dX(t*)/dt = 0 implies that 



NX* 



(3) 



where R* = R(t*). We extrapolate p N (X*) linearly in X and R, that is, 



Pn(X, R) -. 



- P° R +P°Q 
NX* 



R+p° N (X*-X) 



if X<X* + 4^ 



P°n (4) 



otherwise 



where p^ is a positive constant so that Pn(X, _R) > 0 for all X, R S 0. 
Note that the stationarity condition causes the transition rate per 
time unit and forum participant p N to depend on both X and R. 
Because control is exerted by individual recruiters we assume con- 
dition (4) will be applied with a time delay t > 0. This delay is one 
factor that makes it impossible for the recruiter to stay below the 
targetX* indefinitely (see Methods, section "Implications of Delayed 
Control"). 

We also need to take into account that the target set by the recrui- 
ters, X*, cannot be stationary over time, because the lack of attacks on 
the forum raises the expectations of the target values. We model this 
by a linear factor, a, that increases the target value, X* over time. 
Figure 1 shows an example of the exponential growth rate of the 
forum and how it tends to decrease as the number of members 
increase. The equations for the controlled forum can be written as 



dX 
~dt 
dR 
~di 
dO 
~dt 
dX* 
dt 



=p N (X(t-T),R(t-?)X)(N-X-R-O)X-(p 0 R +p° o )R, 
=p° R R, 

=/o«> 

=1, 



(5) 



where p N (X, R, X*) is given as in equation (4). Note that we have 
replaced N in the first equation by (JV - X - _R - O) because the 
number of individuals susceptible of becoming forum members are 
the ones that remain outside the Forum. 

It can be shown analytically that this model with control displays 
the double exponential behavior of the empirical data. There is fast 
exponential growth when forum administrators are not exerting 
control (X(t) =Ae ct — a/c when t ~ t, t > t), and slower exponential 
growth (X(t)=Bel" + q/p = Be pt +X* when t ~ r*) when they are 
close to their control target, i.e. c> p (see Methods, Section "Linear 
asymptotic behavior of the forum"). Moreover, there is additional, 
subtle evidence in the data that such control mechanisms exist - 
notice in Figure 1 that Forum 5 breaks out of an equilibrium around 
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Figure 2 | Fit of the model to the Forum 8 by minimizing root mean 
square error (RMSE) for the model with control (solid black line) and the 
uncontrolled one (solid blue). Note that even when we add an additional 
parameter to the uncontrolled model (the number of susceptible 
individuals, N) the model without control cannot fit the data well 
( dashed line) . 



Day 1450 and becomes exponential again, suggesting that the control 
mechanism was discontinued at that point. 

We also derived an alternative model without control in which the 
only limiting factor for growth of the forum was the size of the 
population N (see Methods, section "Specification of the model with- 
out control"). In other words, it is possible that the slowdown in 
growth may be driven by the fact that there are fewer susceptible 
individuals in the population as forum membership increases. 
However, in Figure 2 we use data from Forum 8 to demonstrate that 
the model with control outperforms the alternative model, even 
when we let N be a free parameter of the model that we can fit. We 
also show in Table 3 the values fitted to other fora in our data set, all 
of which indicate that the controlled model fits considerably better 
than the uncontrolled model. The parameter p^ can be easily esti- 
mated from the equations that describe the initial growth of the 
forum (see Methods, section "Linear asymptotic behavior of the 
forum," CASE 1). The parameters p R and p° 0 can also be uniquely 
estimated by the saturation process of the forum (details in Methods 
section "Linear asymptotic behavior of the forum," CASE 2). The rest 
of the parameters are obtained by minimizing the RMSE using the 
PGAPack library 52 . Note that the mean-field model we propose is 
based on a deterministic (not stochastic) dynamical system, and 
therefore it has a unique solution for a particular set of parameters. 

We now extend our model of a single forum to one that includes 
the interplay between several fora by linking them together in a 
model of competition. Let I be the total number of individuals, 
N(t) the number of individuals susceptible to entering a forum (that 
is, those who have not yet entered a forum at time t), N F the total 
number of fora. For simplicity let us assume that each individual can 
only belong to one forum. Then, 



I 



(6) 



which means that the total number of individuals in the system is 
conserved. When a particular forum j is eliminated at time x, then 
N(t + ) =N(t_) +X(r_)andX(T + ) = 0. Note that this may lead to a 
potential increase in the growth of other active fora. Forum j can 
reappear again (perhaps with a different name or identity) but with a 
lower equilibrium point, adjusted below the level that triggered the 
attack as X*'*(t+) = jiX''*{T-), where can vary in the range (0, 1). 
None of the fora know a priori what the threshold 0 = X(x-) value is, 
and this uncertainty creates a market for competition ruled by self- 
regulation. We model the reappearing time with a Poisson distri- 
bution with rate of 1 event in time T R . 
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Table 3 | Parameter values of the model fitted to each individual forum by minimizing root mean square error (RMSE) for the controlled 
model. For comparison, the RMSE obtained after fitting the parameters of the uncontrolled model is also shown for each forum. Forum 5 is not 
included in the table as it breaks out of an equilibrium around Day 1450 and becomes exponential again, suggesting that the control 
mechanism was discontinued at that point 



Fnnim rslnmp 

1 \J 1 \J 1 1 1 INUII1C? 


RMSE control 

i\l V \\J l_ LUI 1 1 1 \J 1 


RaASF nn-mntrnl 

i\l V \ l_ 1 \\J l_^H 1 1 \J 1 


0° 


0° 

Pr 


0° 
Po 


X* If = 01 




Forum 1 


545 


1239 


2.42-10" 8 


0.002 


0.0004 


2275 


19.58 


Forum 2 


97 


106 


3.87- 10" 8 


0.002 


0.000044 


4153 


0.09 


Forum 3 


85 


171 


3.50- lO" 8 


0.0003 


0.001 


2937 


12.40 


Forum 4 


29 


271 


2.75- 10~ 8 


0.002 


0.0008 


2220 


0.66 


Forum 6 


49 


122 


1.31-1 0 s 


0.005 


0.0004 


7430 


10.38 


Forum 7 


73 


141 


8.08- 10" 8 


0.005 


0.0004 


7423 


10.44 


Forum 8 


436 


1398 


2.66- 10~ 8 


0.0002 


0.0007 


2330 


10.83 


Forum 9 


48 


122 


1 .32-1 0 s 


0.005 


0.0005 


7432 


1 1.57 



Parameter Mean 3.24 -lO" 8 0.0026 0.0006 4525 9.49 

Standard 2.00 -10" 8 0.0020 0.0002 2480 6.37 

Deviation 



We make use of the conservation identity in equation (6) to derive 
the following expressions for rates of growth in each forum i = 1 , . . . , 

N F : 

dX' 

-p N (X i {t--c),R i {t-T)X'*)X i x 



dt 



I- 




dR> 
~dl 




dO 1 
~dt 




dx u 




dt 





{p R +p 0 o )R l , 



(7) 



Initially each forum has zero members, X(r = 0) = 0, and at least 1 
recruiter, R'(t = 0) = 1, and produces no violent output O'(0) = 0. 
Note that each forum fixes their own target values X''*(r = 0), with 
their own estimation of the threshold starting with a uniformly ran- 
dom distribution of thresholds at the beginning. Since fora are 
started at random times, we initialize p N (X'(t — t), R'(t — t), X''*) 
with some probability, k. 

As an illustration we conduct a numerical simulation of 50 fora, 
N P = 50, depicted in Figure 3. An external agent (for example, a 
counter-terrorism agency) allows fora to grow until they reach 
14, 000 members (9 = 14, 000) but no forum administrator knows 
this a priori. In this example, we assume that administrators apply 
their control rules with a one day delay, x = 1. When a forum is 
attacked and deactivated, its membership goes back to 0, and it 
reappears with an adjusted threshold that is fi = 0.99 times the 
former value. We also assume that on average 20 fora can be acti- 
vated in a year, that is 
k = 20/(N P X 365 days). 

The results show that fora are able to control the size of their 
membership for long periods of time, which is not a trivial problem 
from a distributed control point of view. Moreover, differences in 
forum lifetimes are based on the heterogeneity in their different 
control target thresholds, X' *, and the variability in the parameter 
values Pn>Pr> andpQ. This effect leads to different rates of growth for 
different fora, even though the fact that policymakers exert the same 
pressure on all of them. 

To illustrate the implications of self-regulation, we show in 
Figure 4 the dependence of total output and the number of fora 
disrupted as a function of the policy maker threshold, 0 . Notably, 



total output increases nearly exponentially with the threshold, sug- 
gesting that even weak control mechanisms can have a big effect on 
reducing membership. However, the marginal effect of a stricter 
(lower 0) policy decreases dramatically as control is tightened. 
Note also that the number of fora disrupted decreases exponentially 
with 0, suggesting that efforts to tighten control will require an 
increasingly costly effort in detection and deactivation. Therefore, 
the amount of extremism reduced per unit cost of control will decline 
quickly with stricter policies both because of decreasing effectiveness 
of the attacks as well as increasing costs of detection/deactivation of 
fora. 

Discussion 

The model presented here reproduces the observed dynamics of 
violent extremist fora. In particular, the model shows (i) an initially 
steep and exponential increase in forum members, (ii) an intermedi- 
ate interval in which the number of new members continues to grow 
exponentially but at a decreasing rate, and (iii) a final stabilization 
period, in which the number of participants smoothly converges 
(and, eventually, overshoots) a target number of members. Our ana- 
lysis shows that the model is robust to varying assumptions about the 
recruitment and control process, accommodating any arbitrary 
number of status ranks in these fora, and allowing for time delay 
in the implementation of the access control mechanism. 

Even though we and other researchers lack detailed evidence about 
when and how forum self-controls might be implemented, it is 
known that extremist fora are the target of law enforcement organi- 
zations. Our model suggests that a simple strategic reaction to these 
attacks can explain dynamic changes in forum membership. A plaus- 
ible explanation suggested by field- researchers is that the forum 
leaders are able to 'broadcast' their message to the population in a 
top down manner rather than across a well defined network struc- 
ture, which is known to display a bursty, slow speed of diffusion via 
word-of-mouth 53,54 . Our model can also explain oddities in the 
empirical data, such as the observation that some fora stop growing 
for long periods of time, and, in some cases, they suddenly start 
growing after being stagnant (see Figure 1; Forum 5). This suggests 
that some fora occasionally eliminate or drastically alter their self- 
control mechanisms to allow for a new burst of growth. 

The model also lets us test the effect of different strategies that 
might be employed by law enforcement agencies. The fact that mem- 
bership increases exponentially with less strict policies means that 
stricter policies may be decreasingly effective at the margin. 
Moreover, the number of fora that must be attacked increases expo- 
nentially with stricter policies, suggesting that the costs of enforcing 
stricter polices may increase exponentially. Thus, we expect a sharp 
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Figure 3 | Simulated number of members of 25 randomly chosen fora out of 50 during a period of 10 years: J = 17,000,000, Nf= 50 p 1 ^ = 3.24e- 
p R = 0.0026,^ = 0.0006, ff= 14, 000, and r = 1 day, and A = 9.49. 



decline in the effectiveness of targeting additional fora of smaller size. 
As a result, we would recommend targeting policies aimed at occa- 
sionally disrupting a small number of the largest groups. Resources 
devoted to a stricter policy might be better used elsewhere. Our 
findings therefore back Berger's "Strategy doesn't have to be an all- 
or-nothing proposition." 55 . 

An important limitation of our model is that it is just a model - 
there may be other ways to explain the growth rates we observe. Our 
results yield better fit than a simple growth model without control, 
suggesting that the fora have recruitment procedures that control 
access in order to keep the number of members below a time- 
dependent threshold. But it is possible that other processes, particu- 
larly those based on recruitment within social networks (rather than 
random recruitment from the population) may also fit the data. We 
also note that the limited size of our dataset may hinder our efforts to 
validate the proposed model, but it is important to emphasize that 
these data are difficult to collect due ethical, experimental, and legal 
impediments to monitoring the activity of extremist fora. Finally, one 
might expect that a deliberate policy of limiting the rate of growth to 
be something that would itself be discussed on a forum, but during 
our observations we did not find any evidence of such discussions. 

That said, if this model approximates the true underlying process, 
it suggests that even minimal efforts to target extremist fora may have 
a large effect on reducing their membership and therefore their capa- 
city to recruit for real world violent acts. It also suggests that much of 
this effect is due to the self-control induced in the fora rather than the 
direct effect of temporarily dispersing the members of a given forum. 

While our original aim was to better understand extremist groups, 
we note that our model might also be of use to those seeking to 
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Figure 4 | Total output and number of forum disruptions versus the 
policymaker threshold 0 (lower values indicate stricter policies) using the 
same set of parameters as in Figure 3. 



understand how legitimate fora react under stress. For example, there 
have been numerous recent examples of repressive regimes that seek 
to thwart the use of social media to mobilize legitimate opposition 
protest 56-57 and in future work it would be interesting to see whether 
our model can explain changes in the membership and self-control 
mechanisms adopted by fora that are used in such protests. 

Methods 

Forum names and URLs are available from the authors on request. 
Implications of delayed control. The evolution equation for X is 

dX 



dt 



=p N (X(t-T)Mt-T))NX(t) - (p° +p° 0 )R(t) 

^R(t-T)X(t) +p° N N{X> -X(t-T)]X(t) - (j> 0 R +p Q o )R(t) 



forX<X'+p' N /p° N , and 



f = -(f K +P%)R 



(8) 



(9) 



for X >X* fp^- The resulting model yields delayed control behavior that is like 
steering a boat: one tends to overshoot the target values because the action is 
implemented with a delay. 

Since the smaller x the better the control, we will consider the condition expressed 
in equation (8). The target value X* would be reached at time f* by the uncontrolled 
system. At this point, 



R(f 



dR, , 



= (l-q,»)ir + 0(T 2 ) 

by (5), and 

X*-JC(t*-T)=rJ(t*) + 0(x 2 ) 

= zp- N NX*-z(pl+p 0 o )R* + O{z 2 ) 

by equation (2) and equation (3). It follows from equation (8) that up to 0{t 2 ) 
dX 



(10) 



(11) 



dt 



in = (pi +P°o) (1 - vl)R' - (pi +P°o)R' + 0(x 2 ) = -x(A +P%)pW + 0(r 2 ) 



The fact that dX(t*)/dt < 0 for i > 0 sufficiently small, shows that X(t) exceeds X* 

dX 

before the time f = f* (since X(t* -t) ~X* -T — (t*) >X*). 

Linear asymptotic behavior of the forum. Let us linearize at the early stages of the 
formation of the forums and at later stages when forum membership reaches a high 
number. We know from Eq. (5) that 
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R(t)=R 0 e 



Replacing this in Eq. (8), we get 



dX 



-(f) 



jl+Pl 



«„ e «'-"x(f)+p>[X*-X(f-T)]X(0-(rf+p° o )%^'. (12) 



This equation is nonlinear due to the second term on the right hand side. With this 
equation we can now study two limiting cases. 

CASE 1. If t « t, f > t, then we may replace X* - X( t - t) by X* in ( 1 2) since X(f - t) 
«C X*. Eq. (12) becomes linear: 

dX , 



-(t)-P(t)X(t) = Q(t), 



where 



=p° N NX*= : c>0, 
because e^C-'l^l and# 0 /X* < 1, and 

~-(Pk+Po)«o= : -«(fl>0). 

Hence 

X(f)=Ae c '-f . 



(13) 



(14) 



CASE 2. Iff = f* then X* - X(t - 
linear, see Eq. (13), but now 

«o _1_ 

Ht)' 



X* 

P a R +P°o 
X' 



) ~ 0 according to Eq. (11). Eq. (12) becomes also 
"=:p>0, (15) 



Analogously to Case 1, the solution is 

X{t)=Be pt + ^=Be pl +X*. 

P 

This shows that the population dynamics model is consistent wit the double expo- 
nentials observed in the data. Moreover, it follows from Eq. 1 4 that X( t ) starts growing 
at an exponential rate given by the parameter 

c=p° N NX'. 

By the time X(t) < X*, the exponential growth rate implied by Eq. 15 is 

Since N, X* ^> 1, and on general grounds > P°r>Po> we conc l UQ, e that c> p, i.e., the 
growth of forum members is much greater at the beginning than at the end of the time 
span (0, t*). 



Specification of the model without control. The model without control can be 
simply written as 

^=p N (N-X-R-O)X-(pl+p 0 o )R, 
dR „ 



dO 



-p« 0 R. 



These equations show that the only mechanism the uncontrolled system has for 
saturation is the reduction of susceptible individuals {N - X - R - 0) over time. 
However Table 3 and Figure 2 indicate that even when N is small and an adjustable 
parameter, the controlled model fits the empirical data considerably better than the 
uncontrolled model. 
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